MS1-634US 



Prior Art 



102 



I — -— 1 

Source 1 




' Source I 


Splitter 




Filter f° 


Filter 



Transform 
Filter A 



110 



Transform 
Filter B 



108 



100 




3 



Application Program(s) 




COM ^e- 302 
Interfaces 



216 



300 



222 



304A 



306A 



Source 
Filter 



Transform 
Filter(s) 



Source Transform 
Filter ~ [ Filter(s) 

^ 304N V - 306Nr- 



Render Engine 
^308 



Dynamic Matrix 
Switch Filter 



306B 



Transform 
Filters) 



Transform 
Filter(s) 



310 



Render 
Filter 



306C 



From Source (file, AA/ tape, AA/ device.antenna, etc.) 
From Source (file, AA/ tape, AA/ device, antenna, etc.) 



To Hardware 
Renderer 



MS1-634US 




MS1-634US 



402< 



410 



Media Time 



308 



Input 0 



Matrix Switch 



Matrix Switch 
Programming Grid 



Input N 



412 



Input/Output 
Buffer(s) 



Project Time 



Output 0 
406 

-408 



Output N 



>404 



■J 





© 





0 0 





© 



MS1-634US 



o 
o 

CO 



CD 
O 



CD 
O 
LO 



0 

E 
i- 
"6 

CD 

e 1 

CL 



0 
CL 

CO 
C 

o 

o 

CD 

c 
c 

o . 
o 0 

.E E 

CL '-i= 

3 O 
Q. CD 

SB 1 

c 

C 

o 

CL 
CO 

o 
o 



o 5 co 

its 



o 

LO 







C 








de 


CD 


c 


E 











CO 

o 

CO 



CM 
CD 



CD 



CD 



CD 



1_ 




CD 






CD 


8 






Ll 


Q 









o 

CM 
CD 



o 




"ansiti 


<D 


E 


I- 





CO 




o 


0 






Eff 


Ll 



CM 
CM 
CD 



r u q) 

TO > n= 





i_ 




CD 




■ 


O 


E 


FR 


Pack 






ize 


CD 


CO 




Re 


E 




i— 




ode 


i_ 

0 


o 




CD 


E 


Q 





$ 0) 



0 









i— 






o 


E 


CO 





MS1-634US 



'Pty. 7 



Start 

702 



Receive a request to generate media 
processing project utilizing resources 
of filter graph generator 

Identify number of media sources 
selected by user 



Identify required source, transform 
and rendering filter(s) to effect 
processing project 



704 



706 



Based on user editing instructions, /" 
filter graph manager establishes a 
programming grid for matrix switch 
filter, identifying which inputs are to 
be coupled to which outputs at 
particular project times 



Based, at least in part, on the y 
programming grid, filter graph 
generator establishes a matrix switch 
filter and assembles the filter graph 



712 

Filters (re)negotiate buffer memory 
requirements between filters 



Execute media processing project on 
demand 



714 



MS1-634US 



802 



804 



806 



Matrix switch notes the maximum 
buffer requirements for any object 
coupled to any of its inputs/outputs 



Negotiate buffer size and attribute 
characteristics for buffers between 
I/O 



Matrix switch connects all of its I/O to 
adjacent objects with the new buffer 
requirements, attempting to be the 
allocator in all of its pin connections 



712 



No 



810 



Buffer copy operations 
will need to be performed 
from input(s) to output(s) 
for I/O which switch is not 
allocator 



808 

Allocator for all I/O? 




Yes 



812 



Restore connections in remaining 
filters using negotiated buffer 
requirements 



j 



MS1-634US 



^900 

User-defined 
Editing Project 



A 



^910 

Data Structure 
representing 
Project, e.g. a tree 



902 r~ 904 906 



Source j ( Trans. 



\ 



r 



908 



/ \ 
/ \ 
/ \ 




^912 

Data Structure 

Utilized to 
Program Matrix 
Switch, e.g. grid 



' A 
B 

C 

T 



0 






0 


1 






2 


0 




0 





time 0 



r 



908 



8 



10 



12 



14 



16 



B 



T 



MS1-634US 




MS1-634US 



1300 



0 2 4 6 8 10 12 14 16 



0 



t3 



1300 



0 2 4 6 8 10 12 14 16 



A 
B 



0 






0 





1300 



0 2 4 6 8 10 12 14 16 



A 


0 




B 




0 




C 




0 



1300 



0 2 4 6 8 10 12 14 16 



A 


0 




B 




0 




C 




0 


Trans 




0 





16 



MS1-634US 



1300 



0 2 4 6 8 10 12 14 16 



A 


0 




B 




0 


[0]1 




C 




[0]2 


0 


Trans 




0 





7<^. 17 



1300 



yj 

□ 
01 
-I 

nl 
□ 
01 

Ci 

□ 



0 



8 



10 12 14 16 



(0) A 


0 






(1) B 




0 


[0]1 




(2) C 




[0]2 


0 


(3) Trans 




0 





t% 



MS1-634US 



1900 




1902 



Define a first data structure that represents 
an editing project 



1904 



Process first data structure to provide a 
second data structure configured to 
program the matrix switch 



1 906 Program matrix switch using second data 

structure 



MS1-634US 



time 0 2 4 6 8 10 12 14 



B 



Effect 



2000-^ 




MS1-634US 



2200 



0 2 4 6 8 10 12 14 16 



A 


0 




B 




[0]1 




Fx 




0 





2200^. 



0 



8 



10 12 14 16 



□ 

yj 
111 
□ 

m 



(0) A 


0 




(1) B 




[0]1 




(2) Fx 




0 


[0]2 




(3) C 




[0]3 


0 


(4) T 




0 





fli 

□ 
Si 
CI 
O 



A 
B 
FXr 



Trans 



Matrix Switch 

0 

1 

2 
3 



0 



2 
3 



Fx 



Trans — i 



0 Pty.24 



MS1-634US 




^,26 



MS1-634US 



2700 



2702 



(10) A 

(11) B 

(12) C 

(13) D 

(14) E 

(15) Mix 





[0]3 


[0]4 








[0]5 




[0]6 




[0]7 


0 



Matrix Switch 



0 



A 1 

B 1 

C H 

D 1 

E 1 



10 
11 
12 
13 
14 



3 
4 

5 
6 

7 



Mix 



15 



MS1-634US 




MS1-634US 




MS1-634US 



3100 



0 2 4 6 8 10 12 14 16 



A 

(• 



0 


0 


1 






2 


0 




0 






3100 



CI 
g» 

Uf 
ni 
ci 

ill 

□ 

□ 
□ 



0 2 4 6 8 10 12 14 16 



A 

G 

T1 



3100 



(0) A 

(1) /B 

(2) I C 

(3) \T2 

(4) T1 



0 






0 


1 






2 


0 




0 






0 





7^.36 



0 2 4 6 8 10 12 14 16 



0 


3 






4 


0 


1 






2 


0 




0 






0 





MS1-634US 



A 
B 
C 



Matrix Switch 

0 
1 

2 
3 
4 



0 

1 
2 

3 
4 



T2 



T1 



3900 



3902 



3904 



3906 



Define a multimedia editing project to 
include at least one composite that 
represents multiple tracks as a single track 



Define a first data structure that represents 
the editing project 



Process first data structure to provide a 
second data structure configured to 
program a matrix switch 



Program matrix switch using second data 
structure 



7ty 



MS1-634US 



Start 



4002 



I 



Receive indication to generate 
development project 



Identify all sources utilized in development 
project, and when they are required. 



i 



Generate and manage filter graph utilizing 
only those chains for sources which are 
necessary to support current execution of 
the filter graph 



i 



End 



o 
o 



41 

m 
□ 
m 

M 
fil 

Ci 

CI 

Ci 



00 

o 



CD 
O 



CO 
CD 
'O 

c 

CD 
"O 

c 

CD 
Q_ 
CD 
□ 



CD 
E 

i- 

CD 

2 
o 

CO 



CD 



O 
CD 

'2 



O 



CD 

O 
CO 



CM 

o 



c 

'CO 

O 



0 

c 
O 



q 
o 

CO 

c\i 

Csj 

o 
o 
o 
q 
o 



> 

CO 

> 
o 

E 



CM 
O 
CO 



o 

CNJ 

^1- 



CO 
CO 

d 



LO 

CO 

cSi 

CM 
i 

O 

CO 

c\i 



> 

03 

csi 

92 
"> 
o 

E 



o 

CO 



A 

CD 
« 

CD 

■o 

V 



A 

CD 
• 

CD 
"D 

V 



A 
CD 

•4— ' 

CD 
V 



A 
CD 

CD 
"D 

V 



00 
O 
CO 



rr 

CM 



MS1-634US 



42 



From 4004 



r 



4006 



I 



Identify chain(s) required in the next M 
seconds 



4202 



4204 



Y 



Load chain(s) 



4206 




4208 



Identify currently loaded 
chain(s) that will not be used 
in the next N seconds 



4210 




B 



Unload identified 
thread(s) 



4212 

Chain-count<V^>^-f A 





MS1-634US 







c 






u 


de 


0 


c 


il 


CD 











1— 




0 






O 


il 


FR 


Pack 



0 , 

CO ±i 



i— 




rse 


Iter 


Pa 


Ll 




CO 




o 


0 


0 




Eff 


LL 



« £ - 









0 






O 


il 


FR 


Pack 



i 



0 

N 0 

CO ±i 

0 lT 



^ CO 



Si 



o 





0 




o 

I— 


I— 

0 


Z3 


-i— » 


o 


il 


CO 





CO 

o 

CO 

^1- 




<o 

O CD 

qz ll 
LU 




O 
O 



J= £ & 

m > — 



0 





1 0> 




! S 


0 






i O 


il 


j CO 





CO 
O 
CO 




CO "> == 

5 65"- 



£ 0 

CO ±± 



£ 0 

CO ±S 



i 



